{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7.4 word2vec、fastTextを用いた日本語単語のベクトル表現の実装\n",
    "\n",
    "- 本ファイルでは、日本語の単語をword2vecもしくはfastTextを使用してベクトル化する手法を解説します。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "※　本章のファイルはすべてUbuntuでの動作を前提としています。Windowsなど文字コードが違う環境での動作にはご注意下さい。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7.4 学習目標\n",
    "\n",
    "1.\t学習済みの日本語word2vecモデルで単語をベクトル表現に変換する実装ができるようになる\n",
    "2.\t学習済みの日本語fastText モデルで単語をベクトル表現に変換する実装ができるようになる\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 事前準備\n",
    "書籍の指示に従い、本章で使用するデータを用意します\n",
    "\n",
    "pip install gensim\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. 文書を読み込んで、分かち書き、データセット作成まで（8.2と同じです）\n",
    "\n",
    "前処理と分かち書きをし、最後にデータセットを作成する部分を実装します\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 単語分割にはMecab＋NEologdを使用\n",
    "import MeCab\n",
    "\n",
    "m_t = MeCab.Tagger('-Owakati -d /usr/lib/mecab/dic/mecab-ipadic-neologd')\n",
    "\n",
    "def tokenizer_mecab(text):\n",
    "    text = m_t.parse(text)  # これでスペースで単語が区切られる\n",
    "    ret = text.strip().split()  # スペース部分で区切ったリストに変換\n",
    "    return ret\n",
    "\n",
    "\n",
    "\n",
    "# 前処理として正規化をする関数を定義\n",
    "import re\n",
    "\n",
    "def preprocessing_text(text):\n",
    "    # 改行、半角スペース、全角スペースを削除\n",
    "    text = re.sub('\\r', '', text)\n",
    "    text = re.sub('\\n', '', text)\n",
    "    text = re.sub('　', '', text)\n",
    "    text = re.sub(' ', '', text)\n",
    "\n",
    "    # 数字文字の一律「0」化\n",
    "    text = re.sub(r'[0-9 ０-９]', '0', text)  # 数字\n",
    "\n",
    "    return text\n",
    "\n",
    "\n",
    "# 前処理とJanomeの単語分割を合わせた関数を定義する\n",
    "\n",
    "\n",
    "def tokenizer_with_preprocessing(text):\n",
    "    text = preprocessing_text(text)  # 前処理の正規化\n",
    "    ret = tokenizer_mecab(text)  # Mecabの単語分割\n",
    "\n",
    "    return ret\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torchtext\n",
    "\n",
    "# tsvやcsvデータを読み込んだときに、読み込んだ内容に対して行う処理を定義します\n",
    "# 文章とラベルの両方に用意します\n",
    "\n",
    "max_length = 25\n",
    "TEXT = torchtext.data.Field(sequential=True, tokenize=tokenizer_with_preprocessing,\n",
    "                            use_vocab=True, lower=True, include_lengths=True, batch_first=True, fix_length=max_length)\n",
    "LABEL = torchtext.data.Field(sequential=False, use_vocab=False)\n",
    "\n",
    "\n",
    "# フォルダ「data」から各tsvファイルを読み込みます\n",
    "train_ds, val_ds, test_ds = torchtext.data.TabularDataset.splits(\n",
    "    path='./data/', train='text_train.tsv',\n",
    "    validation='text_val.tsv', test='text_test.tsv', format='tsv',\n",
    "    fields=[('Text', TEXT), ('Label', LABEL)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 単語のベクトル化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 word2vec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "単語をベクトル表現に変換します。\n",
    "\n",
    "TorchTextには日本語の学習済みデータがないわけではないですが、精度が微妙なので\n",
    "\n",
    "東北大学 乾・岡崎研究室で公開されているWord2Vecの学習済みのベクトルを使用します。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 以下から、日本語のfasttextの学習済みベクトルをダウンロードします\n",
    "\n",
    "# 東北大学 乾・岡崎研究室：日本語 Wikipedia エンティティベクトル\n",
    "\n",
    "# http://www.cl.ecei.tohoku.ac.jp/~m-suzuki/jawiki_vector/\n",
    "# http://www.cl.ecei.tohoku.ac.jp/~m-suzuki/jawiki_vector/data/20170201.tar.bz2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/smart_open/smart_open_lib.py:398: UserWarning: This function is deprecated, use smart_open.open instead. See the migration notes for details: https://github.com/RaRe-Technologies/smart_open/blob/master/README.rst#migrating-to-the-new-open-function\n",
      "  'See the migration notes for details: %s' % _MIGRATION_NOTES_URL\n",
      "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/ipykernel_launcher.py:15: DeprecationWarning: Call to deprecated `wv` (Attribute will be removed in 4.0.0, use self instead).\n",
      "  from ipykernel import kernelapp as app\n",
      "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/smart_open/smart_open_lib.py:398: UserWarning: This function is deprecated, use smart_open.open instead. See the migration notes for details: https://github.com/RaRe-Technologies/smart_open/blob/master/README.rst#migrating-to-the-new-open-function\n",
      "  'See the migration notes for details: %s' % _MIGRATION_NOTES_URL\n"
     ]
    }
   ],
   "source": [
    "# そのままではtorchtextで読み込めないので、gensimライブラリを使用して、\n",
    "# Word2Vecのformatで保存し直します\n",
    "\n",
    "# 事前インストール\n",
    "# pip install gensim\n",
    "\n",
    "from gensim.models import KeyedVectors\n",
    "\n",
    "\n",
    "# 一度gensimライブラリで読み込んで、word2vecのformatで保存する\n",
    "model = KeyedVectors.load_word2vec_format(\n",
    "    './data/entity_vector/entity_vector.model.bin', binary=True)\n",
    "\n",
    "# 保存（時間がかかります、10分弱）\n",
    "model.wv.save_word2vec_format('./data/japanese_word2vec_vectors.vec')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 0/1015474 [00:00<?, ?it/s]Skipping token b'1015474' with 1-dimensional vector [b'200']; likely a header\n",
      "100%|█████████▉| 1014564/1015474 [01:45<00:00, 9573.88it/s] "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1単語を表現する次元数： 200\n",
      "単語数： 1015474\n"
     ]
    }
   ],
   "source": [
    "# torchtextで単語ベクトルとして読み込みます\n",
    "from torchtext.vocab import Vectors\n",
    "\n",
    "japanese_word2vec_vectors = Vectors(\n",
    "    name='./data/japanese_word2vec_vectors.vec')\n",
    "\n",
    "# 単語ベクトルの中身を確認します\n",
    "print(\"1単語を表現する次元数：\", japanese_word2vec_vectors.dim)\n",
    "print(\"単語数：\", len(japanese_word2vec_vectors.itos))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([49, 200])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],\n",
       "        [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],\n",
       "        [ 2.6023, -2.6357, -2.5822,  ...,  0.6953, -1.4977,  1.4752],\n",
       "        ...,\n",
       "        [-2.8353,  2.5609, -0.5348,  ...,  0.4602,  1.4669, -2.1255],\n",
       "        [-1.5885,  0.1614, -0.6029,  ..., -1.7545, -1.2462,  2.3034],\n",
       "        [-0.0448, -0.1304,  0.0329,  ...,  0.0825, -0.1386,  0.0417]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "100%|█████████▉| 1014564/1015474 [02:00<00:00, 9573.88it/s]"
     ]
    }
   ],
   "source": [
    "# ベクトル化したバージョンのボキャブラリーを作成します\n",
    "TEXT.build_vocab(train_ds, vectors=japanese_word2vec_vectors, min_freq=1)\n",
    "\n",
    "# ボキャブラリーのベクトルを確認します\n",
    "print(TEXT.vocab.vectors.shape)  # 49個の単語が200次元のベクトルで表現されている\n",
    "TEXT.vocab.vectors\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defaultdict(<function torchtext.vocab._default_unk_index()>,\n",
       "            {'<unk>': 0,\n",
       "             '<pad>': 1,\n",
       "             'と': 2,\n",
       "             '。': 3,\n",
       "             'な': 4,\n",
       "             'の': 5,\n",
       "             '文章': 6,\n",
       "             '、': 7,\n",
       "             'が': 8,\n",
       "             'し': 9,\n",
       "             'を': 10,\n",
       "             'いる': 11,\n",
       "             'か': 12,\n",
       "             'て': 13,\n",
       "             'ます': 14,\n",
       "             '分類': 15,\n",
       "             '本章': 16,\n",
       "             '評価': 17,\n",
       "             '0': 18,\n",
       "             'い': 19,\n",
       "             'から': 20,\n",
       "             'する': 21,\n",
       "             'その': 22,\n",
       "             'た': 23,\n",
       "             'で': 24,\n",
       "             'です': 25,\n",
       "             'に': 26,\n",
       "             'に対して': 27,\n",
       "             'は': 28,\n",
       "             'まし': 29,\n",
       "             'クラス': 30,\n",
       "             'ネガティブ': 31,\n",
       "             'ポジティブ': 32,\n",
       "             'モデル': 33,\n",
       "             'レビュー': 34,\n",
       "             '値': 35,\n",
       "             '取り組み': 36,\n",
       "             '商品': 37,\n",
       "             '女性': 38,\n",
       "             '女王': 39,\n",
       "             '好き': 40,\n",
       "             '姫': 41,\n",
       "             '構築': 42,\n",
       "             '機械学習': 43,\n",
       "             '王': 44,\n",
       "             '王子': 45,\n",
       "             '男性': 46,\n",
       "             '短い': 47,\n",
       "             '自然言語処理': 48})"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# ボキャブラリーの単語の順番を確認します\n",
    "TEXT.vocab.stoi\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "女王 tensor(0.3840)\n",
      "王 tensor(0.3669)\n",
      "王子 tensor(0.5489)\n",
      "機械学習 tensor(-0.1404)\n"
     ]
    }
   ],
   "source": [
    "# 姫 - 女性 + 男性 のベクトルがどれと似ているのか確認してみます\n",
    "import torch.nn.functional as F\n",
    "\n",
    "# 姫 - 女性 + 男性\n",
    "tensor_calc = TEXT.vocab.vectors[41] - \\\n",
    "    TEXT.vocab.vectors[38] + TEXT.vocab.vectors[46]\n",
    "\n",
    "# コサイン類似度を計算\n",
    "# dim=0 は0次元目で計算してくださいという指定\n",
    "print(\"女王\", F.cosine_similarity(tensor_calc, TEXT.vocab.vectors[39], dim=0))\n",
    "print(\"王\", F.cosine_similarity(tensor_calc, TEXT.vocab.vectors[44], dim=0))\n",
    "print(\"王子\", F.cosine_similarity(tensor_calc, TEXT.vocab.vectors[45], dim=0))\n",
    "print(\"機械学習\", F.cosine_similarity(tensor_calc, TEXT.vocab.vectors[43], dim=0))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "姫 - 女性 + 男性　を計算すると狙った通り、王子がもっとも近い結果になりました"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 fastText"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "word2vecより進歩したベクトル化手法であるfastTextによる単語のベクトル表現を使用します。\n",
    "\n",
    "日本語の学習モデルを以下の記事にて公開してくださっているので、使用させていただきます。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Qiita：いますぐ使える単語埋め込みベクトルのリスト\n",
    "# https://qiita.com/Hironsan/items/8f7d35f0a36e0f99752c\n",
    "\n",
    "# Download Word Vectors\n",
    "# https://drive.google.com/open?id=0ByFQ96A4DgSPNFdleG1GaHcxQzA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "  0%|          | 0/351122 [00:00<?, ?it/s]\u001b[ASkipping token b'351122' with 1-dimensional vector [b'300']; likely a header\n",
      "\n",
      "  0%|          | 441/351122 [00:00<01:19, 4400.59it/s]\u001b[A\n",
      "  0%|          | 875/351122 [00:00<01:19, 4381.08it/s]\u001b[A\n",
      "  0%|          | 1603/351122 [00:00<01:10, 4974.78it/s]\u001b[A\n",
      "  1%|          | 2343/351122 [00:00<01:03, 5516.17it/s]\u001b[A\n",
      "  1%|          | 3163/351122 [00:00<00:56, 6116.46it/s]\u001b[A\n",
      "  1%|          | 3994/351122 [00:00<00:52, 6641.58it/s]\u001b[A\n",
      "  1%|▏         | 4804/351122 [00:00<00:49, 7018.58it/s]\u001b[A\n",
      "  2%|▏         | 5636/351122 [00:00<00:46, 7363.78it/s]\u001b[A\n",
      "  2%|▏         | 6388/351122 [00:00<00:46, 7407.18it/s]\u001b[A\n",
      "  2%|▏         | 7206/351122 [00:01<00:45, 7621.03it/s]\u001b[A\n",
      "  2%|▏         | 7973/351122 [00:01<00:45, 7584.77it/s]\u001b[A\n",
      "  2%|▏         | 8735/351122 [00:01<00:45, 7492.27it/s]\u001b[A\n",
      "  3%|▎         | 9487/351122 [00:01<00:45, 7439.72it/s]\u001b[A\n",
      "  3%|▎         | 10233/351122 [00:01<00:45, 7411.51it/s]\u001b[A\n",
      "  3%|▎         | 10976/351122 [00:01<00:46, 7373.08it/s]\u001b[A\n",
      "  3%|▎         | 11715/351122 [00:01<00:46, 7365.82it/s]\u001b[A\n",
      "  4%|▎         | 12453/351122 [00:01<00:46, 7348.57it/s]\u001b[A\n",
      "  4%|▍         | 13189/351122 [00:01<00:45, 7351.46it/s]\u001b[A\n",
      "  4%|▍         | 13925/351122 [00:01<00:45, 7339.56it/s]\u001b[A\n",
      "  4%|▍         | 14661/351122 [00:02<00:45, 7342.91it/s]\u001b[A\n",
      "  4%|▍         | 15396/351122 [00:02<00:45, 7333.41it/s]\u001b[A\n",
      "  5%|▍         | 16130/351122 [00:02<00:45, 7334.10it/s]\u001b[A\n",
      "  5%|▍         | 16864/351122 [00:02<00:45, 7310.33it/s]\u001b[A\n",
      "  5%|▌         | 17612/351122 [00:02<00:45, 7360.17it/s]\u001b[A\n",
      "  5%|▌         | 18355/351122 [00:02<00:45, 7377.99it/s]\u001b[A\n",
      "  5%|▌         | 19105/351122 [00:02<00:44, 7412.31it/s]\u001b[A\n",
      "  6%|▌         | 19847/351122 [00:02<00:44, 7388.61it/s]\u001b[A\n",
      "  6%|▌         | 20589/351122 [00:02<00:44, 7396.77it/s]\u001b[A\n",
      "  6%|▌         | 21334/351122 [00:02<00:44, 7412.10it/s]\u001b[A\n",
      "  6%|▋         | 22076/351122 [00:03<00:46, 7116.61it/s]\u001b[A\n",
      "  6%|▋         | 22802/351122 [00:03<00:45, 7158.89it/s]\u001b[A\n",
      "  7%|▋         | 23548/351122 [00:03<00:45, 7244.71it/s]\u001b[A\n",
      "  7%|▋         | 24275/351122 [00:03<00:45, 7232.27it/s]\u001b[A\n",
      "  7%|▋         | 25007/351122 [00:03<00:44, 7256.18it/s]\u001b[A\n",
      "  7%|▋         | 25742/351122 [00:03<00:44, 7282.73it/s]\u001b[A\n",
      "  8%|▊         | 26476/351122 [00:03<00:44, 7297.95it/s]\u001b[A\n",
      "  8%|▊         | 27213/351122 [00:03<00:44, 7316.51it/s]\u001b[A\n",
      "  8%|▊         | 27945/351122 [00:03<00:44, 7303.41it/s]\u001b[A\n",
      "  8%|▊         | 28676/351122 [00:03<00:44, 7301.69it/s]\u001b[A\n",
      "  8%|▊         | 29407/351122 [00:04<00:44, 7290.25it/s]\u001b[A\n",
      "  9%|▊         | 30146/351122 [00:04<00:43, 7317.83it/s]\u001b[A\n",
      "  9%|▉         | 30878/351122 [00:04<00:43, 7308.48it/s]\u001b[A\n",
      "  9%|▉         | 31609/351122 [00:04<00:43, 7301.39it/s]\u001b[A\n",
      "  9%|▉         | 32340/351122 [00:04<00:43, 7301.06it/s]\u001b[A\n",
      "  9%|▉         | 33071/351122 [00:04<00:43, 7284.87it/s]\u001b[A\n",
      " 10%|▉         | 33808/351122 [00:04<00:43, 7307.39it/s]\u001b[A\n",
      " 10%|▉         | 34539/351122 [00:04<00:43, 7264.88it/s]\u001b[A\n",
      " 10%|█         | 35268/351122 [00:04<00:43, 7272.36it/s]\u001b[A\n",
      " 10%|█         | 35996/351122 [00:04<00:43, 7265.32it/s]\u001b[A\n",
      " 10%|█         | 36826/351122 [00:05<00:41, 7545.40it/s]\u001b[A\n",
      " 11%|█         | 37584/351122 [00:05<00:42, 7431.67it/s]\u001b[A\n",
      " 11%|█         | 38330/351122 [00:05<00:42, 7385.98it/s]\u001b[A\n",
      " 11%|█         | 39071/351122 [00:05<00:42, 7345.97it/s]\u001b[A\n",
      " 11%|█▏        | 39807/351122 [00:05<00:42, 7342.23it/s]\u001b[A\n",
      " 12%|█▏        | 40543/351122 [00:05<00:43, 7076.84it/s]\u001b[A\n",
      " 12%|█▏        | 41254/351122 [00:05<00:44, 6921.32it/s]\u001b[A\n",
      " 12%|█▏        | 41949/351122 [00:05<00:45, 6860.52it/s]\u001b[A\n",
      " 12%|█▏        | 42660/351122 [00:05<00:44, 6932.07it/s]\u001b[A\n",
      " 12%|█▏        | 43490/351122 [00:05<00:42, 7291.60it/s]\u001b[A\n",
      " 13%|█▎        | 44237/351122 [00:06<00:41, 7342.70it/s]\u001b[A\n",
      " 13%|█▎        | 44988/351122 [00:06<00:41, 7389.56it/s]\u001b[A\n",
      " 13%|█▎        | 45800/351122 [00:06<00:40, 7593.63it/s]\u001b[A\n",
      " 13%|█▎        | 46563/351122 [00:06<00:40, 7506.49it/s]\u001b[A\n",
      " 13%|█▎        | 47317/351122 [00:06<00:41, 7360.64it/s]\u001b[A\n",
      " 14%|█▎        | 48056/351122 [00:06<00:41, 7361.65it/s]\u001b[A\n",
      " 14%|█▍        | 48804/351122 [00:06<00:40, 7396.53it/s]\u001b[A\n",
      " 14%|█▍        | 49545/351122 [00:06<00:40, 7359.49it/s]\u001b[A\n",
      " 14%|█▍        | 50369/351122 [00:06<00:39, 7601.47it/s]\u001b[A\n",
      " 15%|█▍        | 51173/351122 [00:06<00:38, 7725.33it/s]\u001b[A\n",
      " 15%|█▍        | 52008/351122 [00:07<00:37, 7902.07it/s]\u001b[A\n",
      " 15%|█▌        | 52843/351122 [00:07<00:37, 8029.71it/s]\u001b[A\n",
      " 15%|█▌        | 53680/351122 [00:07<00:36, 8127.33it/s]\u001b[A\n",
      " 16%|█▌        | 54516/351122 [00:07<00:36, 8193.90it/s]\u001b[A\n",
      " 16%|█▌        | 55337/351122 [00:07<00:37, 7958.42it/s]\u001b[A\n",
      " 16%|█▌        | 56136/351122 [00:07<00:37, 7796.90it/s]\u001b[A\n",
      " 16%|█▌        | 56919/351122 [00:07<00:38, 7710.50it/s]\u001b[A\n",
      " 16%|█▋        | 57693/351122 [00:07<00:38, 7524.32it/s]\u001b[A\n",
      " 17%|█▋        | 58514/351122 [00:07<00:37, 7717.33it/s]\u001b[A\n",
      " 17%|█▋        | 59343/351122 [00:08<00:37, 7878.37it/s]\u001b[A\n",
      " 17%|█▋        | 60134/351122 [00:08<00:37, 7739.94it/s]\u001b[A\n",
      " 17%|█▋        | 60911/351122 [00:08<00:37, 7643.14it/s]\u001b[A\n",
      " 18%|█▊        | 61738/351122 [00:08<00:37, 7820.86it/s]\u001b[A\n",
      " 18%|█▊        | 62570/351122 [00:08<00:36, 7963.12it/s]\u001b[A\n",
      " 18%|█▊        | 63401/351122 [00:08<00:35, 8062.88it/s]\u001b[A\n",
      " 18%|█▊        | 64210/351122 [00:08<00:36, 7878.26it/s]\u001b[A\n",
      " 19%|█▊        | 65034/351122 [00:08<00:35, 7981.49it/s]\u001b[A\n",
      " 19%|█▉        | 65848/351122 [00:08<00:35, 8027.56it/s]\u001b[A\n",
      " 19%|█▉        | 66653/351122 [00:08<00:36, 7806.11it/s]\u001b[A\n",
      " 19%|█▉        | 67436/351122 [00:09<00:37, 7660.51it/s]\u001b[A\n",
      " 19%|█▉        | 68205/351122 [00:09<00:37, 7586.26it/s]\u001b[A\n",
      " 20%|█▉        | 68966/351122 [00:09<00:37, 7502.76it/s]\u001b[A\n",
      " 20%|█▉        | 69718/351122 [00:09<00:37, 7435.00it/s]\u001b[A\n",
      " 20%|██        | 70463/351122 [00:09<00:38, 7384.37it/s]\u001b[A\n",
      " 20%|██        | 71205/351122 [00:09<00:37, 7393.92it/s]\u001b[A\n",
      " 20%|██        | 71946/351122 [00:09<00:37, 7357.07it/s]\u001b[A\n",
      " 21%|██        | 72683/351122 [00:09<00:39, 7087.60it/s]\u001b[A\n",
      " 21%|██        | 73395/351122 [00:09<00:39, 7001.37it/s]\u001b[A\n",
      " 21%|██        | 74098/351122 [00:09<00:40, 6897.00it/s]\u001b[A\n",
      " 21%|██▏       | 74790/351122 [00:10<00:40, 6808.99it/s]\u001b[A\n",
      " 21%|██▏       | 75473/351122 [00:10<00:40, 6743.23it/s]\u001b[A\n",
      " 22%|██▏       | 76149/351122 [00:10<00:41, 6693.15it/s]\u001b[A\n",
      " 22%|██▏       | 76820/351122 [00:10<00:41, 6657.54it/s]\u001b[A\n",
      " 22%|██▏       | 77487/351122 [00:10<00:41, 6642.39it/s]\u001b[A\n",
      " 22%|██▏       | 78152/351122 [00:10<00:41, 6642.72it/s]\u001b[A\n",
      " 22%|██▏       | 78817/351122 [00:10<00:41, 6638.11it/s]\u001b[A\n",
      " 23%|██▎       | 79482/351122 [00:10<00:40, 6628.01it/s]\u001b[A\n",
      " 23%|██▎       | 80149/351122 [00:10<00:40, 6639.01it/s]\u001b[A\n",
      " 23%|██▎       | 80814/351122 [00:11<00:40, 6615.96it/s]\u001b[A\n",
      " 23%|██▎       | 81476/351122 [00:11<00:40, 6606.29it/s]\u001b[A\n",
      " 23%|██▎       | 82137/351122 [00:11<00:40, 6579.31it/s]\u001b[A\n",
      " 24%|██▎       | 82796/351122 [00:11<00:40, 6554.06it/s]\u001b[A\n",
      " 24%|██▍       | 83454/351122 [00:11<00:40, 6561.26it/s]\u001b[A\n",
      " 24%|██▍       | 84111/351122 [00:11<00:40, 6552.91it/s]\u001b[A\n",
      " 24%|██▍       | 84773/351122 [00:11<00:40, 6571.83it/s]\u001b[A\n",
      " 24%|██▍       | 85434/351122 [00:11<00:40, 6580.65it/s]\u001b[A\n",
      " 25%|██▍       | 86093/351122 [00:11<00:40, 6556.60it/s]\u001b[A\n",
      " 25%|██▍       | 86752/351122 [00:11<00:40, 6564.67it/s]\u001b[A\n",
      " 25%|██▍       | 87413/351122 [00:12<00:40, 6577.47it/s]\u001b[A\n",
      " 25%|██▌       | 88071/351122 [00:12<00:40, 6561.02it/s]\u001b[A\n",
      " 25%|██▌       | 88731/351122 [00:12<00:39, 6570.48it/s]\u001b[A\n",
      " 25%|██▌       | 89389/351122 [00:12<00:39, 6564.20it/s]\u001b[A\n",
      " 26%|██▌       | 90046/351122 [00:12<00:39, 6564.13it/s]\u001b[A\n",
      " 26%|██▌       | 90706/351122 [00:12<00:39, 6574.41it/s]\u001b[A\n",
      " 26%|██▌       | 91365/351122 [00:12<00:39, 6578.80it/s]\u001b[A\n",
      " 26%|██▌       | 92027/351122 [00:12<00:39, 6590.62it/s]\u001b[A\n",
      " 26%|██▋       | 92689/351122 [00:12<00:39, 6597.26it/s]\u001b[A\n",
      " 27%|██▋       | 93349/351122 [00:12<00:39, 6593.99it/s]\u001b[A\n",
      " 27%|██▋       | 94009/351122 [00:13<00:39, 6554.78it/s]\u001b[A\n",
      " 27%|██▋       | 94670/351122 [00:13<00:39, 6570.40it/s]\u001b[A\n",
      " 27%|██▋       | 95328/351122 [00:13<00:39, 6540.13it/s]\u001b[A\n",
      " 27%|██▋       | 95987/351122 [00:13<00:38, 6553.97it/s]\u001b[A\n",
      " 28%|██▊       | 96643/351122 [00:13<00:38, 6550.35it/s]\u001b[A\n",
      " 28%|██▊       | 97299/351122 [00:13<00:39, 6507.52it/s]\u001b[A\n",
      " 28%|██▊       | 97950/351122 [00:13<00:38, 6494.52it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 28%|██▊       | 98600/351122 [00:13<00:38, 6475.84it/s]\u001b[A\n",
      " 28%|██▊       | 99262/351122 [00:13<00:38, 6518.09it/s]\u001b[A\n",
      " 28%|██▊       | 99925/351122 [00:13<00:38, 6549.91it/s]\u001b[A\n",
      " 29%|██▊       | 100586/351122 [00:14<00:38, 6567.06it/s]\u001b[A\n",
      " 29%|██▉       | 101246/351122 [00:14<00:37, 6576.73it/s]\u001b[A\n",
      " 29%|██▉       | 101904/351122 [00:14<00:38, 6536.44it/s]\u001b[A\n",
      " 29%|██▉       | 102562/351122 [00:14<00:37, 6547.37it/s]\u001b[A\n",
      " 29%|██▉       | 103223/351122 [00:14<00:37, 6563.10it/s]\u001b[A\n",
      " 30%|██▉       | 103880/351122 [00:14<00:37, 6561.42it/s]\u001b[A\n",
      " 30%|██▉       | 104539/351122 [00:14<00:37, 6569.41it/s]\u001b[A\n",
      " 30%|██▉       | 105198/351122 [00:14<00:37, 6573.33it/s]\u001b[A\n",
      " 30%|███       | 105856/351122 [00:14<00:37, 6574.13it/s]\u001b[A\n",
      " 30%|███       | 106514/351122 [00:14<00:37, 6573.16it/s]\u001b[A\n",
      " 31%|███       | 107172/351122 [00:15<00:37, 6474.66it/s]\u001b[A\n",
      " 31%|███       | 107833/351122 [00:15<00:37, 6512.47it/s]\u001b[A\n",
      " 31%|███       | 108485/351122 [00:15<00:37, 6478.89it/s]\u001b[A\n",
      " 31%|███       | 109146/351122 [00:15<00:37, 6514.43it/s]\u001b[A\n",
      " 31%|███▏      | 109806/351122 [00:15<00:36, 6539.55it/s]\u001b[A\n",
      " 31%|███▏      | 110464/351122 [00:15<00:36, 6550.96it/s]\u001b[A\n",
      " 32%|███▏      | 111120/351122 [00:15<00:36, 6540.13it/s]\u001b[A\n",
      " 32%|███▏      | 111781/351122 [00:15<00:36, 6558.34it/s]\u001b[A\n",
      " 32%|███▏      | 112437/351122 [00:15<00:36, 6551.52it/s]\u001b[A\n",
      " 32%|███▏      | 113098/351122 [00:15<00:36, 6568.50it/s]\u001b[A\n",
      " 32%|███▏      | 113757/351122 [00:16<00:36, 6573.07it/s]\u001b[A\n",
      " 33%|███▎      | 114415/351122 [00:16<00:36, 6563.91it/s]\u001b[A\n",
      " 33%|███▎      | 115073/351122 [00:16<00:35, 6567.70it/s]\u001b[A\n",
      " 33%|███▎      | 115730/351122 [00:16<00:35, 6555.63it/s]\u001b[A\n",
      " 33%|███▎      | 116386/351122 [00:16<00:35, 6554.11it/s]\u001b[A\n",
      " 33%|███▎      | 117042/351122 [00:16<00:35, 6519.64it/s]\u001b[A\n",
      " 34%|███▎      | 117695/351122 [00:16<00:35, 6484.08it/s]\u001b[A\n",
      " 34%|███▎      | 118353/351122 [00:16<00:35, 6511.66it/s]\u001b[A\n",
      " 34%|███▍      | 119005/351122 [00:16<00:35, 6476.67it/s]\u001b[A\n",
      " 34%|███▍      | 119662/351122 [00:16<00:35, 6502.58it/s]\u001b[A\n",
      " 34%|███▍      | 120313/351122 [00:17<00:35, 6472.83it/s]\u001b[A\n",
      " 34%|███▍      | 120964/351122 [00:17<00:35, 6481.53it/s]\u001b[A\n",
      " 35%|███▍      | 121623/351122 [00:17<00:35, 6512.83it/s]\u001b[A\n",
      " 35%|███▍      | 122283/351122 [00:17<00:34, 6538.47it/s]\u001b[A\n",
      " 35%|███▌      | 122941/351122 [00:17<00:34, 6549.07it/s]\u001b[A\n",
      " 35%|███▌      | 123596/351122 [00:17<00:34, 6526.14it/s]\u001b[A\n",
      " 35%|███▌      | 124257/351122 [00:17<00:34, 6550.61it/s]\u001b[A\n",
      " 36%|███▌      | 124919/351122 [00:17<00:34, 6569.50it/s]\u001b[A\n",
      " 36%|███▌      | 125577/351122 [00:17<00:34, 6567.84it/s]\u001b[A\n",
      " 36%|███▌      | 126249/351122 [00:17<00:34, 6597.50it/s]\u001b[A\n",
      " 36%|███▌      | 126909/351122 [00:18<00:34, 6576.53it/s]\u001b[A\n",
      " 36%|███▋      | 127567/351122 [00:18<00:34, 6542.69it/s]\u001b[A\n",
      " 37%|███▋      | 128222/351122 [00:18<00:34, 6530.83it/s]\u001b[A\n",
      " 37%|███▋      | 128876/351122 [00:18<00:34, 6491.28it/s]\u001b[A\n",
      " 37%|███▋      | 129526/351122 [00:18<00:34, 6395.64it/s]\u001b[A\n",
      " 37%|███▋      | 130166/351122 [00:18<00:34, 6362.23it/s]\u001b[A\n",
      " 37%|███▋      | 130818/351122 [00:18<00:34, 6407.02it/s]\u001b[A\n",
      " 37%|███▋      | 131475/351122 [00:18<00:34, 6453.85it/s]\u001b[A\n",
      " 38%|███▊      | 132125/351122 [00:18<00:33, 6466.22it/s]\u001b[A\n",
      " 38%|███▊      | 132772/351122 [00:18<00:34, 6378.27it/s]\u001b[A\n",
      " 38%|███▊      | 133430/351122 [00:19<00:33, 6435.89it/s]\u001b[A\n",
      " 38%|███▊      | 134090/351122 [00:19<00:33, 6482.05it/s]\u001b[A\n",
      " 38%|███▊      | 134747/351122 [00:19<00:33, 6506.03it/s]\u001b[A\n",
      " 39%|███▊      | 135405/351122 [00:19<00:33, 6526.61it/s]\u001b[A\n",
      " 39%|███▉      | 136061/351122 [00:19<00:32, 6534.11it/s]\u001b[A\n",
      " 39%|███▉      | 136717/351122 [00:19<00:32, 6538.81it/s]\u001b[A\n",
      " 39%|███▉      | 137379/351122 [00:19<00:32, 6560.03it/s]\u001b[A\n",
      " 39%|███▉      | 138040/351122 [00:19<00:32, 6572.06it/s]\u001b[A\n",
      " 40%|███▉      | 138698/351122 [00:19<00:32, 6569.23it/s]\u001b[A\n",
      " 40%|███▉      | 139355/351122 [00:19<00:32, 6565.60it/s]\u001b[A\n",
      " 40%|███▉      | 140012/351122 [00:20<00:32, 6536.86it/s]\u001b[A\n",
      " 40%|████      | 140666/351122 [00:20<00:32, 6530.09it/s]\u001b[A\n",
      " 40%|████      | 141320/351122 [00:20<00:32, 6525.30it/s]\u001b[A\n",
      " 40%|████      | 141975/351122 [00:20<00:32, 6529.76it/s]\u001b[A\n",
      " 41%|████      | 142633/351122 [00:20<00:31, 6541.76it/s]\u001b[A\n",
      " 41%|████      | 143291/351122 [00:20<00:31, 6551.56it/s]\u001b[A\n",
      " 41%|████      | 143951/351122 [00:20<00:31, 6562.96it/s]\u001b[A\n",
      " 41%|████      | 144608/351122 [00:20<00:31, 6529.82it/s]\u001b[A\n",
      " 41%|████▏     | 145262/351122 [00:20<00:31, 6511.58it/s]\u001b[A\n",
      " 42%|████▏     | 145914/351122 [00:20<00:31, 6510.69it/s]\u001b[A\n",
      " 42%|████▏     | 146574/351122 [00:21<00:31, 6535.90it/s]\u001b[A\n",
      " 42%|████▏     | 147235/351122 [00:21<00:31, 6556.01it/s]\u001b[A\n",
      " 42%|████▏     | 147896/351122 [00:21<00:30, 6571.05it/s]\u001b[A\n",
      " 42%|████▏     | 148558/351122 [00:21<00:30, 6584.83it/s]\u001b[A\n",
      " 42%|████▏     | 149218/351122 [00:21<00:30, 6587.61it/s]\u001b[A\n",
      " 43%|████▎     | 149880/351122 [00:21<00:30, 6595.26it/s]\u001b[A\n",
      " 43%|████▎     | 150541/351122 [00:21<00:30, 6599.41it/s]\u001b[A\n",
      " 43%|████▎     | 151201/351122 [00:21<00:30, 6597.69it/s]\u001b[A\n",
      " 43%|████▎     | 151862/351122 [00:21<00:30, 6598.62it/s]\u001b[A\n",
      " 43%|████▎     | 152522/351122 [00:21<00:30, 6571.14it/s]\u001b[A\n",
      " 44%|████▎     | 153180/351122 [00:22<00:30, 6549.15it/s]\u001b[A\n",
      " 44%|████▍     | 153840/351122 [00:22<00:30, 6563.89it/s]\u001b[A\n",
      " 44%|████▍     | 154498/351122 [00:22<00:29, 6568.66it/s]\u001b[A\n",
      " 44%|████▍     | 155157/351122 [00:22<00:29, 6573.22it/s]\u001b[A\n",
      " 44%|████▍     | 155815/351122 [00:22<00:29, 6563.54it/s]\u001b[A\n",
      " 45%|████▍     | 156476/351122 [00:22<00:29, 6577.07it/s]\u001b[A\n",
      " 45%|████▍     | 157135/351122 [00:22<00:29, 6579.57it/s]\u001b[A\n",
      " 45%|████▍     | 157795/351122 [00:22<00:29, 6584.90it/s]\u001b[A\n",
      " 45%|████▌     | 158456/351122 [00:22<00:29, 6592.33it/s]\u001b[A\n",
      " 45%|████▌     | 159120/351122 [00:22<00:29, 6603.86it/s]\u001b[A\n",
      " 46%|████▌     | 159781/351122 [00:23<00:28, 6602.88it/s]\u001b[A\n",
      " 46%|████▌     | 160442/351122 [00:23<00:28, 6580.23it/s]\u001b[A\n",
      " 46%|████▌     | 161101/351122 [00:23<00:29, 6508.51it/s]\u001b[A\n",
      " 46%|████▌     | 161753/351122 [00:23<00:29, 6481.30it/s]\u001b[A\n",
      " 46%|████▋     | 162411/351122 [00:23<00:28, 6510.16it/s]\u001b[A\n",
      " 46%|████▋     | 163071/351122 [00:23<00:28, 6535.72it/s]\u001b[A\n",
      " 47%|████▋     | 163725/351122 [00:23<00:28, 6515.05it/s]\u001b[A\n",
      " 47%|████▋     | 164383/351122 [00:23<00:28, 6533.97it/s]\u001b[A\n",
      " 47%|████▋     | 165037/351122 [00:23<00:28, 6504.53it/s]\u001b[A\n",
      " 47%|████▋     | 165694/351122 [00:23<00:28, 6522.73it/s]\u001b[A\n",
      " 47%|████▋     | 166355/351122 [00:24<00:28, 6547.86it/s]\u001b[A\n",
      " 48%|████▊     | 167016/351122 [00:24<00:28, 6563.54it/s]\u001b[A\n",
      " 48%|████▊     | 167675/351122 [00:24<00:27, 6570.76it/s]\u001b[A\n",
      " 48%|████▊     | 168336/351122 [00:24<00:27, 6580.91it/s]\u001b[A\n",
      " 48%|████▊     | 168995/351122 [00:24<00:27, 6558.12it/s]\u001b[A\n",
      " 48%|████▊     | 169655/351122 [00:24<00:27, 6568.33it/s]\u001b[A\n",
      " 49%|████▊     | 170312/351122 [00:24<00:27, 6558.28it/s]\u001b[A\n",
      " 49%|████▊     | 170972/351122 [00:24<00:27, 6569.52it/s]\u001b[A\n",
      " 49%|████▉     | 171630/351122 [00:24<00:27, 6570.71it/s]\u001b[A\n",
      " 49%|████▉     | 172292/351122 [00:24<00:27, 6583.41it/s]\u001b[A\n",
      " 49%|████▉     | 172951/351122 [00:25<00:27, 6521.42it/s]\u001b[A\n",
      " 49%|████▉     | 173608/351122 [00:25<00:27, 6533.86it/s]\u001b[A\n",
      " 50%|████▉     | 174262/351122 [00:25<00:27, 6515.26it/s]\u001b[A\n",
      " 50%|████▉     | 174914/351122 [00:25<00:27, 6502.38it/s]\u001b[A\n",
      " 50%|█████     | 175574/351122 [00:25<00:26, 6529.56it/s]\u001b[A\n",
      " 50%|█████     | 176229/351122 [00:25<00:26, 6532.35it/s]\u001b[A\n",
      " 50%|█████     | 176883/351122 [00:25<00:26, 6527.21it/s]\u001b[A\n",
      " 51%|█████     | 177536/351122 [00:25<00:26, 6521.92it/s]\u001b[A\n",
      " 51%|█████     | 178190/351122 [00:25<00:26, 6527.05it/s]\u001b[A\n",
      " 51%|█████     | 178843/351122 [00:26<00:26, 6478.78it/s]\u001b[A\n",
      " 51%|█████     | 179491/351122 [00:26<00:26, 6476.19it/s]\u001b[A\n",
      " 51%|█████▏    | 180139/351122 [00:26<00:26, 6465.15it/s]\u001b[A\n",
      " 51%|█████▏    | 180795/351122 [00:26<00:26, 6490.36it/s]\u001b[A\n",
      " 52%|█████▏    | 181452/351122 [00:26<00:26, 6513.99it/s]\u001b[A\n",
      " 52%|█████▏    | 182107/351122 [00:26<00:25, 6521.82it/s]\u001b[A\n",
      " 52%|█████▏    | 182760/351122 [00:26<00:26, 6439.47it/s]\u001b[A\n",
      " 52%|█████▏    | 183405/351122 [00:26<00:26, 6431.18it/s]\u001b[A\n",
      " 52%|█████▏    | 184051/351122 [00:26<00:25, 6439.21it/s]\u001b[A\n",
      " 53%|█████▎    | 184696/351122 [00:26<00:25, 6438.85it/s]\u001b[A\n",
      " 53%|█████▎    | 185359/351122 [00:27<00:25, 6492.73it/s]\u001b[A\n",
      " 53%|█████▎    | 186013/351122 [00:27<00:25, 6506.48it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 53%|█████▎    | 186664/351122 [00:27<00:25, 6505.21it/s]\u001b[A\n",
      " 53%|█████▎    | 187325/351122 [00:27<00:25, 6533.74it/s]\u001b[A\n",
      " 54%|█████▎    | 187986/351122 [00:27<00:24, 6554.24it/s]\u001b[A\n",
      " 54%|█████▎    | 188644/351122 [00:27<00:24, 6560.54it/s]\u001b[A\n",
      " 54%|█████▍    | 189301/351122 [00:27<00:24, 6553.02it/s]\u001b[A\n",
      " 54%|█████▍    | 189957/351122 [00:27<00:24, 6528.32it/s]\u001b[A\n",
      " 54%|█████▍    | 190614/351122 [00:27<00:24, 6539.69it/s]\u001b[A\n",
      " 54%|█████▍    | 191269/351122 [00:27<00:24, 6529.26it/s]\u001b[A\n",
      " 55%|█████▍    | 191923/351122 [00:28<00:24, 6530.32it/s]\u001b[A\n",
      " 55%|█████▍    | 192582/351122 [00:28<00:24, 6545.42it/s]\u001b[A\n",
      " 55%|█████▌    | 193242/351122 [00:28<00:24, 6560.68it/s]\u001b[A\n",
      " 55%|█████▌    | 193900/351122 [00:28<00:23, 6565.31it/s]\u001b[A\n",
      " 55%|█████▌    | 194559/351122 [00:28<00:23, 6571.23it/s]\u001b[A\n",
      " 56%|█████▌    | 195217/351122 [00:28<00:23, 6568.06it/s]\u001b[A\n",
      " 56%|█████▌    | 195878/351122 [00:28<00:23, 6579.04it/s]\u001b[A\n",
      " 56%|█████▌    | 196536/351122 [00:28<00:23, 6506.59it/s]\u001b[A\n",
      " 56%|█████▌    | 197187/351122 [00:28<00:23, 6418.01it/s]\u001b[A\n",
      " 56%|█████▋    | 197843/351122 [00:28<00:23, 6457.29it/s]\u001b[A\n",
      " 57%|█████▋    | 198502/351122 [00:29<00:23, 6496.35it/s]\u001b[A\n",
      " 57%|█████▋    | 199155/351122 [00:29<00:23, 6503.82it/s]\u001b[A\n",
      " 57%|█████▋    | 199807/351122 [00:29<00:23, 6506.46it/s]\u001b[A\n",
      " 57%|█████▋    | 200464/351122 [00:29<00:23, 6524.63it/s]\u001b[A\n",
      " 57%|█████▋    | 201121/351122 [00:29<00:22, 6535.57it/s]\u001b[A\n",
      " 57%|█████▋    | 201775/351122 [00:29<00:22, 6535.11it/s]\u001b[A\n",
      " 58%|█████▊    | 202429/351122 [00:29<00:22, 6535.79it/s]\u001b[A\n",
      " 58%|█████▊    | 203083/351122 [00:29<00:22, 6510.91it/s]\u001b[A\n",
      " 58%|█████▊    | 203739/351122 [00:29<00:22, 6524.17it/s]\u001b[A\n",
      " 58%|█████▊    | 204394/351122 [00:29<00:22, 6530.46it/s]\u001b[A\n",
      " 58%|█████▊    | 205050/351122 [00:30<00:22, 6537.23it/s]\u001b[A\n",
      " 59%|█████▊    | 205704/351122 [00:30<00:22, 6521.99it/s]\u001b[A\n",
      " 59%|█████▉    | 206361/351122 [00:30<00:22, 6535.19it/s]\u001b[A\n",
      " 59%|█████▉    | 207015/351122 [00:30<00:22, 6517.60it/s]\u001b[A\n",
      " 59%|█████▉    | 207667/351122 [00:30<00:22, 6507.38it/s]\u001b[A\n",
      " 59%|█████▉    | 208318/351122 [00:30<00:21, 6502.79it/s]\u001b[A\n",
      " 60%|█████▉    | 208969/351122 [00:30<00:21, 6495.80it/s]\u001b[A\n",
      " 60%|█████▉    | 209619/351122 [00:30<00:21, 6478.96it/s]\u001b[A\n",
      " 60%|█████▉    | 210273/351122 [00:30<00:21, 6495.54it/s]\u001b[A\n",
      " 60%|██████    | 210931/351122 [00:30<00:21, 6518.34it/s]\u001b[A\n",
      " 60%|██████    | 211590/351122 [00:31<00:21, 6538.36it/s]\u001b[A\n",
      " 60%|██████    | 212246/351122 [00:31<00:21, 6543.77it/s]\u001b[A\n",
      " 61%|██████    | 212901/351122 [00:31<00:21, 6532.94it/s]\u001b[A\n",
      " 61%|██████    | 213557/351122 [00:31<00:21, 6538.40it/s]\u001b[A\n",
      " 61%|██████    | 214211/351122 [00:31<00:21, 6502.64it/s]\u001b[A\n",
      " 61%|██████    | 214862/351122 [00:31<00:21, 6454.36it/s]\u001b[A\n",
      " 61%|██████▏   | 215508/351122 [00:31<00:21, 6430.21it/s]\u001b[A\n",
      " 62%|██████▏   | 216155/351122 [00:31<00:20, 6440.87it/s]\u001b[A\n",
      " 62%|██████▏   | 216801/351122 [00:31<00:20, 6444.44it/s]\u001b[A\n",
      " 62%|██████▏   | 217455/351122 [00:31<00:20, 6471.32it/s]\u001b[A\n",
      " 62%|██████▏   | 218112/351122 [00:32<00:20, 6498.83it/s]\u001b[A\n",
      " 62%|██████▏   | 218762/351122 [00:32<00:20, 6469.62it/s]\u001b[A\n",
      " 62%|██████▏   | 219412/351122 [00:32<00:20, 6475.80it/s]\u001b[A\n",
      " 63%|██████▎   | 220060/351122 [00:32<00:20, 6474.42it/s]\u001b[A\n",
      " 63%|██████▎   | 220714/351122 [00:32<00:20, 6493.00it/s]\u001b[A\n",
      " 63%|██████▎   | 221366/351122 [00:32<00:19, 6498.89it/s]\u001b[A\n",
      " 63%|██████▎   | 222016/351122 [00:32<00:19, 6475.92it/s]\u001b[A\n",
      " 63%|██████▎   | 222672/351122 [00:32<00:19, 6499.06it/s]\u001b[A\n",
      " 64%|██████▎   | 223329/351122 [00:32<00:19, 6520.09it/s]\u001b[A\n",
      " 64%|██████▍   | 223985/351122 [00:32<00:19, 6529.71it/s]\u001b[A\n",
      " 64%|██████▍   | 224639/351122 [00:33<00:19, 6531.09it/s]\u001b[A\n",
      " 64%|██████▍   | 225294/351122 [00:33<00:19, 6535.63it/s]\u001b[A\n",
      " 64%|██████▍   | 225952/351122 [00:33<00:19, 6546.52it/s]\u001b[A\n",
      " 65%|██████▍   | 226607/351122 [00:33<00:19, 6535.90it/s]\u001b[A\n",
      " 65%|██████▍   | 227261/351122 [00:33<00:18, 6523.11it/s]\u001b[A\n",
      " 65%|██████▍   | 227919/351122 [00:33<00:18, 6538.99it/s]\u001b[A\n",
      " 65%|██████▌   | 228576/351122 [00:33<00:18, 6548.25it/s]\u001b[A\n",
      " 65%|██████▌   | 229231/351122 [00:33<00:18, 6532.18it/s]\u001b[A\n",
      " 65%|██████▌   | 229885/351122 [00:33<00:18, 6499.28it/s]\u001b[A\n",
      " 66%|██████▌   | 230536/351122 [00:33<00:18, 6500.82it/s]\u001b[A\n",
      " 66%|██████▌   | 231196/351122 [00:34<00:18, 6528.84it/s]\u001b[A\n",
      " 66%|██████▌   | 231853/351122 [00:34<00:18, 6539.87it/s]\u001b[A\n",
      " 66%|██████▌   | 232509/351122 [00:34<00:18, 6544.96it/s]\u001b[A\n",
      " 66%|██████▋   | 233166/351122 [00:34<00:18, 6551.35it/s]\u001b[A\n",
      " 67%|██████▋   | 233822/351122 [00:34<00:17, 6519.32it/s]\u001b[A\n",
      " 67%|██████▋   | 234479/351122 [00:34<00:17, 6531.86it/s]\u001b[A\n",
      " 67%|██████▋   | 235133/351122 [00:34<00:17, 6533.86it/s]\u001b[A\n",
      " 67%|██████▋   | 235787/351122 [00:34<00:17, 6499.82it/s]\u001b[A\n",
      " 67%|██████▋   | 236438/351122 [00:34<00:17, 6483.70it/s]\u001b[A\n",
      " 68%|██████▊   | 237087/351122 [00:34<00:17, 6484.23it/s]\u001b[A\n",
      " 68%|██████▊   | 237744/351122 [00:35<00:17, 6509.01it/s]\u001b[A\n",
      " 68%|██████▊   | 238400/351122 [00:35<00:17, 6523.61it/s]\u001b[A\n",
      " 68%|██████▊   | 239059/351122 [00:35<00:17, 6540.94it/s]\u001b[A\n",
      " 68%|██████▊   | 239714/351122 [00:35<00:17, 6494.86it/s]\u001b[A\n",
      " 68%|██████▊   | 240366/351122 [00:35<00:17, 6500.56it/s]\u001b[A\n",
      " 69%|██████▊   | 241017/351122 [00:35<00:16, 6485.89it/s]\u001b[A\n",
      " 69%|██████▉   | 241679/351122 [00:35<00:16, 6523.83it/s]\u001b[A\n",
      " 69%|██████▉   | 242332/351122 [00:35<00:16, 6514.61it/s]\u001b[A\n",
      " 69%|██████▉   | 242991/351122 [00:35<00:16, 6534.08it/s]\u001b[A\n",
      " 69%|██████▉   | 243647/351122 [00:35<00:16, 6539.66it/s]\u001b[A\n",
      " 70%|██████▉   | 244302/351122 [00:36<00:16, 6526.28it/s]\u001b[A\n",
      " 70%|██████▉   | 244963/351122 [00:36<00:16, 6548.14it/s]\u001b[A\n",
      " 70%|██████▉   | 245618/351122 [00:36<00:16, 6526.57it/s]\u001b[A\n",
      " 70%|███████   | 246276/351122 [00:36<00:16, 6539.51it/s]\u001b[A\n",
      " 70%|███████   | 246930/351122 [00:36<00:15, 6521.91it/s]\u001b[A\n",
      " 71%|███████   | 247583/351122 [00:36<00:15, 6517.76it/s]\u001b[A\n",
      " 71%|███████   | 248238/351122 [00:36<00:15, 6525.98it/s]\u001b[A\n",
      " 71%|███████   | 248891/351122 [00:36<00:15, 6522.54it/s]\u001b[A\n",
      " 71%|███████   | 249544/351122 [00:36<00:15, 6500.67it/s]\u001b[A\n",
      " 71%|███████▏  | 250200/351122 [00:36<00:15, 6517.39it/s]\u001b[A\n",
      " 71%|███████▏  | 250861/351122 [00:37<00:15, 6543.76it/s]\u001b[A\n",
      " 72%|███████▏  | 251518/351122 [00:37<00:15, 6551.57it/s]\u001b[A\n",
      " 72%|███████▏  | 252177/351122 [00:37<00:15, 6561.11it/s]\u001b[A\n",
      " 72%|███████▏  | 252834/351122 [00:37<00:14, 6558.83it/s]\u001b[A\n",
      " 72%|███████▏  | 253490/351122 [00:37<00:14, 6557.52it/s]\u001b[A\n",
      " 72%|███████▏  | 254146/351122 [00:37<00:14, 6547.68it/s]\u001b[A\n",
      " 73%|███████▎  | 254801/351122 [00:37<00:14, 6544.20it/s]\u001b[A\n",
      " 73%|███████▎  | 255456/351122 [00:37<00:14, 6545.62it/s]\u001b[A\n",
      " 73%|███████▎  | 256111/351122 [00:37<00:14, 6538.08it/s]\u001b[A\n",
      " 73%|███████▎  | 256765/351122 [00:37<00:14, 6531.69it/s]\u001b[A\n",
      " 73%|███████▎  | 257425/351122 [00:38<00:14, 6551.37it/s]\u001b[A\n",
      " 74%|███████▎  | 258081/351122 [00:38<00:14, 6522.88it/s]\u001b[A\n",
      " 74%|███████▎  | 258743/351122 [00:38<00:14, 6550.44it/s]\u001b[A\n",
      " 74%|███████▍  | 259403/351122 [00:38<00:13, 6563.16it/s]\u001b[A\n",
      " 74%|███████▍  | 260065/351122 [00:38<00:13, 6578.16it/s]\u001b[A\n",
      " 74%|███████▍  | 260728/351122 [00:38<00:13, 6592.12it/s]\u001b[A\n",
      " 74%|███████▍  | 261388/351122 [00:38<00:13, 6570.22it/s]\u001b[A\n",
      " 75%|███████▍  | 262050/351122 [00:38<00:13, 6583.90it/s]\u001b[A\n",
      " 75%|███████▍  | 262713/351122 [00:38<00:13, 6594.76it/s]\u001b[A\n",
      " 75%|███████▌  | 263377/351122 [00:38<00:13, 6605.78it/s]\u001b[A\n",
      " 75%|███████▌  | 264040/351122 [00:39<00:13, 6612.47it/s]\u001b[A\n",
      " 75%|███████▌  | 264702/351122 [00:39<00:13, 6605.40it/s]\u001b[A\n",
      " 76%|███████▌  | 265363/351122 [00:39<00:12, 6600.29it/s]\u001b[A\n",
      " 76%|███████▌  | 266024/351122 [00:39<00:12, 6595.44it/s]\u001b[A\n",
      " 76%|███████▌  | 266684/351122 [00:39<00:12, 6575.91it/s]\u001b[A\n",
      " 76%|███████▌  | 267342/351122 [00:39<00:12, 6574.13it/s]\u001b[A\n",
      " 76%|███████▋  | 268000/351122 [00:39<00:12, 6554.34it/s]\u001b[A\n",
      " 77%|███████▋  | 268656/351122 [00:39<00:12, 6552.41it/s]\u001b[A\n",
      " 77%|███████▋  | 269313/351122 [00:39<00:12, 6556.65it/s]\u001b[A\n",
      " 77%|███████▋  | 269969/351122 [00:39<00:12, 6544.16it/s]\u001b[A\n",
      " 77%|███████▋  | 270624/351122 [00:40<00:12, 6523.06it/s]\u001b[A\n",
      " 77%|███████▋  | 271277/351122 [00:40<00:12, 6505.76it/s]\u001b[A\n",
      " 77%|███████▋  | 271929/351122 [00:40<00:12, 6507.60it/s]\u001b[A\n",
      " 78%|███████▊  | 272584/351122 [00:40<00:12, 6517.75it/s]\u001b[A\n",
      " 78%|███████▊  | 273241/351122 [00:40<00:11, 6531.51it/s]\u001b[A\n",
      " 78%|███████▊  | 273895/351122 [00:40<00:11, 6532.98it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 78%|███████▊  | 274549/351122 [00:40<00:11, 6519.53it/s]\u001b[A\n",
      " 78%|███████▊  | 275201/351122 [00:40<00:11, 6513.50it/s]\u001b[A\n",
      " 79%|███████▊  | 275855/351122 [00:40<00:11, 6520.02it/s]\u001b[A\n",
      " 79%|███████▉  | 276510/351122 [00:40<00:11, 6527.96it/s]\u001b[A\n",
      " 79%|███████▉  | 277163/351122 [00:41<00:11, 6456.57it/s]\u001b[A\n",
      " 79%|███████▉  | 277822/351122 [00:41<00:11, 6495.90it/s]\u001b[A\n",
      " 79%|███████▉  | 278479/351122 [00:41<00:11, 6517.51it/s]\u001b[A\n",
      " 79%|███████▉  | 279141/351122 [00:41<00:10, 6546.93it/s]\u001b[A\n",
      " 80%|███████▉  | 279798/351122 [00:41<00:10, 6552.61it/s]\u001b[A\n",
      " 80%|███████▉  | 280454/351122 [00:41<00:10, 6551.39it/s]\u001b[A\n",
      " 80%|████████  | 281110/351122 [00:41<00:11, 6360.10it/s]\u001b[A\n",
      " 80%|████████  | 281836/351122 [00:41<00:10, 6603.38it/s]\u001b[A\n",
      " 80%|████████  | 282555/351122 [00:41<00:10, 6766.64it/s]\u001b[A\n",
      " 81%|████████  | 283235/351122 [00:41<00:10, 6699.10it/s]\u001b[A\n",
      " 81%|████████  | 283908/351122 [00:42<00:10, 6639.55it/s]\u001b[A\n",
      " 81%|████████  | 284574/351122 [00:42<00:10, 6621.88it/s]\u001b[A\n",
      " 81%|████████  | 285238/351122 [00:42<00:09, 6606.06it/s]\u001b[A\n",
      " 81%|████████▏ | 285900/351122 [00:42<00:09, 6595.73it/s]\u001b[A\n",
      " 82%|████████▏ | 286561/351122 [00:42<00:09, 6575.33it/s]\u001b[A\n",
      " 82%|████████▏ | 287283/351122 [00:42<00:09, 6755.32it/s]\u001b[A\n",
      " 82%|████████▏ | 288004/351122 [00:42<00:09, 6883.56it/s]\u001b[A\n",
      " 82%|████████▏ | 288730/351122 [00:42<00:08, 6992.18it/s]\u001b[A\n",
      " 82%|████████▏ | 289454/351122 [00:42<00:08, 7062.45it/s]\u001b[A\n",
      " 83%|████████▎ | 290188/351122 [00:42<00:08, 7141.96it/s]\u001b[A\n",
      " 83%|████████▎ | 290904/351122 [00:43<00:08, 6953.37it/s]\u001b[A\n",
      " 83%|████████▎ | 291602/351122 [00:43<00:08, 6842.08it/s]\u001b[A\n",
      " 83%|████████▎ | 292288/351122 [00:43<00:08, 6682.49it/s]\u001b[A\n",
      " 83%|████████▎ | 292959/351122 [00:43<00:08, 6637.50it/s]\u001b[A\n",
      " 84%|████████▎ | 293625/351122 [00:43<00:08, 6600.62it/s]\u001b[A\n",
      " 84%|████████▍ | 294287/351122 [00:43<00:08, 6530.38it/s]\u001b[A\n",
      " 84%|████████▍ | 294945/351122 [00:43<00:08, 6542.66it/s]\u001b[A\n",
      " 84%|████████▍ | 295659/351122 [00:43<00:08, 6710.39it/s]\u001b[A\n",
      " 84%|████████▍ | 296382/351122 [00:43<00:07, 6857.16it/s]\u001b[A\n",
      " 85%|████████▍ | 297134/351122 [00:44<00:07, 7042.21it/s]\u001b[A\n",
      " 85%|████████▍ | 297933/351122 [00:44<00:07, 7300.26it/s]\u001b[A\n",
      " 85%|████████▌ | 298668/351122 [00:44<00:07, 7078.82it/s]\u001b[A\n",
      " 85%|████████▌ | 299381/351122 [00:44<00:07, 6933.73it/s]\u001b[A\n",
      " 85%|████████▌ | 300079/351122 [00:44<00:07, 6819.41it/s]\u001b[A\n",
      " 86%|████████▌ | 300764/351122 [00:44<00:07, 6734.18it/s]\u001b[A\n",
      " 86%|████████▌ | 301440/351122 [00:44<00:07, 6680.50it/s]\u001b[A\n",
      " 86%|████████▌ | 302110/351122 [00:44<00:07, 6639.16it/s]\u001b[A\n",
      " 86%|████████▌ | 302831/351122 [00:44<00:07, 6800.02it/s]\u001b[A\n",
      " 86%|████████▋ | 303517/351122 [00:44<00:06, 6816.58it/s]\u001b[A\n",
      " 87%|████████▋ | 304247/351122 [00:45<00:06, 6952.88it/s]\u001b[A\n",
      " 87%|████████▋ | 304979/351122 [00:45<00:06, 7059.02it/s]\u001b[A\n",
      " 87%|████████▋ | 305721/351122 [00:45<00:06, 7161.49it/s]\u001b[A\n",
      " 87%|████████▋ | 306460/351122 [00:45<00:06, 7225.88it/s]\u001b[A\n",
      " 87%|████████▋ | 307201/351122 [00:45<00:06, 7279.56it/s]\u001b[A\n",
      " 88%|████████▊ | 307941/351122 [00:45<00:05, 7314.61it/s]\u001b[A\n",
      " 88%|████████▊ | 308681/351122 [00:45<00:05, 7338.52it/s]\u001b[A\n",
      " 88%|████████▊ | 309423/351122 [00:45<00:05, 7362.16it/s]\u001b[A\n",
      " 88%|████████▊ | 310160/351122 [00:45<00:05, 7343.13it/s]\u001b[A\n",
      " 89%|████████▊ | 310895/351122 [00:45<00:05, 7309.67it/s]\u001b[A\n",
      " 89%|████████▉ | 311627/351122 [00:46<00:05, 7311.46it/s]\u001b[A\n",
      " 89%|████████▉ | 312361/351122 [00:46<00:05, 7317.40it/s]\u001b[A\n",
      " 89%|████████▉ | 313111/351122 [00:46<00:05, 7368.95it/s]\u001b[A\n",
      " 89%|████████▉ | 313917/351122 [00:46<00:04, 7563.38it/s]\u001b[A\n",
      " 90%|████████▉ | 314747/351122 [00:46<00:04, 7770.02it/s]\u001b[A\n",
      " 90%|████████▉ | 315577/351122 [00:46<00:04, 7921.32it/s]\u001b[A\n",
      " 90%|█████████ | 316406/351122 [00:46<00:04, 8026.42it/s]\u001b[A\n",
      " 90%|█████████ | 317211/351122 [00:46<00:04, 7517.24it/s]\u001b[A\n",
      " 91%|█████████ | 317971/351122 [00:46<00:04, 7204.07it/s]\u001b[A\n",
      " 91%|█████████ | 318700/351122 [00:47<00:04, 7002.36it/s]\u001b[A\n",
      " 91%|█████████ | 319408/351122 [00:47<00:04, 6840.24it/s]\u001b[A\n",
      " 91%|█████████ | 320098/351122 [00:47<00:04, 6824.63it/s]\u001b[A\n",
      " 91%|█████████▏| 320824/351122 [00:47<00:04, 6949.16it/s]\u001b[A\n",
      " 92%|█████████▏| 321560/351122 [00:47<00:04, 7066.41it/s]\u001b[A\n",
      " 92%|█████████▏| 322285/351122 [00:47<00:04, 7120.31it/s]\u001b[A\n",
      " 92%|█████████▏| 323018/351122 [00:47<00:03, 7181.58it/s]\u001b[A\n",
      " 92%|█████████▏| 323744/351122 [00:47<00:03, 7204.04it/s]\u001b[A\n",
      " 92%|█████████▏| 324473/351122 [00:47<00:03, 7229.51it/s]\u001b[A\n",
      " 93%|█████████▎| 325201/351122 [00:47<00:03, 7243.48it/s]\u001b[A\n",
      " 93%|█████████▎| 325926/351122 [00:48<00:03, 7129.90it/s]\u001b[A\n",
      " 93%|█████████▎| 326657/351122 [00:48<00:03, 7181.45it/s]\u001b[A\n",
      " 93%|█████████▎| 327394/351122 [00:48<00:03, 7234.75it/s]\u001b[A\n",
      " 93%|█████████▎| 328127/351122 [00:48<00:03, 7261.93it/s]\u001b[A\n",
      " 94%|█████████▎| 328854/351122 [00:48<00:03, 7257.34it/s]\u001b[A\n",
      " 94%|█████████▍| 329595/351122 [00:48<00:02, 7300.96it/s]\u001b[A\n",
      " 94%|█████████▍| 330332/351122 [00:48<00:02, 7319.96it/s]\u001b[A\n",
      " 94%|█████████▍| 331065/351122 [00:48<00:02, 7314.21it/s]\u001b[A\n",
      " 94%|█████████▍| 331803/351122 [00:48<00:02, 7333.69it/s]\u001b[A\n",
      " 95%|█████████▍| 332537/351122 [00:48<00:02, 7319.06it/s]\u001b[A\n",
      " 95%|█████████▍| 333269/351122 [00:49<00:02, 7314.08it/s]\u001b[A\n",
      " 95%|█████████▌| 334002/351122 [00:49<00:02, 7317.45it/s]\u001b[A\n",
      " 95%|█████████▌| 334735/351122 [00:49<00:02, 7320.03it/s]\u001b[A\n",
      " 96%|█████████▌| 335468/351122 [00:49<00:02, 7295.43it/s]\u001b[A\n",
      " 96%|█████████▌| 336201/351122 [00:49<00:02, 7305.46it/s]\u001b[A\n",
      " 96%|█████████▌| 336942/351122 [00:49<00:01, 7334.04it/s]\u001b[A\n",
      " 96%|█████████▌| 337679/351122 [00:49<00:01, 7342.40it/s]\u001b[A\n",
      " 96%|█████████▋| 338414/351122 [00:49<00:01, 7312.15it/s]\u001b[A\n",
      " 97%|█████████▋| 339152/351122 [00:49<00:01, 7331.93it/s]\u001b[A\n",
      " 97%|█████████▋| 339886/351122 [00:49<00:01, 7308.60it/s]\u001b[A\n",
      " 97%|█████████▋| 340622/351122 [00:50<00:01, 7322.92it/s]\u001b[A\n",
      " 97%|█████████▋| 341355/351122 [00:50<00:01, 7314.67it/s]\u001b[A\n",
      " 97%|█████████▋| 342090/351122 [00:50<00:01, 7322.71it/s]\u001b[A\n",
      " 98%|█████████▊| 342823/351122 [00:50<00:01, 7296.63it/s]\u001b[A\n",
      " 98%|█████████▊| 343553/351122 [00:50<00:01, 7275.12it/s]\u001b[A\n",
      " 98%|█████████▊| 344281/351122 [00:50<00:00, 7273.76it/s]\u001b[A\n",
      " 98%|█████████▊| 345009/351122 [00:50<00:00, 7042.05it/s]\u001b[A\n",
      " 98%|█████████▊| 345715/351122 [00:50<00:00, 6886.50it/s]\u001b[A\n",
      " 99%|█████████▊| 346406/351122 [00:50<00:00, 6783.88it/s]\u001b[A\n",
      " 99%|█████████▉| 347087/351122 [00:50<00:00, 6733.91it/s]\u001b[A\n",
      " 99%|█████████▉| 347814/351122 [00:51<00:00, 6884.82it/s]\u001b[A\n",
      " 99%|█████████▉| 348530/351122 [00:51<00:00, 6964.37it/s]\u001b[A\n",
      " 99%|█████████▉| 349258/351122 [00:51<00:00, 7055.96it/s]\u001b[A\n",
      "100%|█████████▉| 349968/351122 [00:51<00:00, 7069.08it/s]\u001b[A\n",
      "100%|█████████▉| 350697/351122 [00:51<00:00, 7133.80it/s]\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1単語を表現する次元数： 300\n",
      "単語数： 351122\n"
     ]
    }
   ],
   "source": [
    "# torchtextで単語ベクトルとして読み込みます\n",
    "# word2vecとは異なり、すぐに読み込めます\n",
    "\n",
    "from torchtext.vocab import Vectors\n",
    "\n",
    "japanese_fasttext_vectors = Vectors(name='./data/vector_neologd/model.vec')\n",
    "\n",
    "                                    \n",
    "# 単語ベクトルの中身を確認します\n",
    "print(\"1単語を表現する次元数：\", japanese_fasttext_vectors.dim)\n",
    "print(\"単語数：\", len(japanese_fasttext_vectors.itos))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([49, 300])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "defaultdict(<function torchtext.vocab._default_unk_index()>,\n",
       "            {'<unk>': 0,\n",
       "             '<pad>': 1,\n",
       "             'と': 2,\n",
       "             '。': 3,\n",
       "             'な': 4,\n",
       "             'の': 5,\n",
       "             '文章': 6,\n",
       "             '、': 7,\n",
       "             'が': 8,\n",
       "             'し': 9,\n",
       "             'を': 10,\n",
       "             'いる': 11,\n",
       "             'か': 12,\n",
       "             'て': 13,\n",
       "             'ます': 14,\n",
       "             '分類': 15,\n",
       "             '本章': 16,\n",
       "             '評価': 17,\n",
       "             '0': 18,\n",
       "             'い': 19,\n",
       "             'から': 20,\n",
       "             'する': 21,\n",
       "             'その': 22,\n",
       "             'た': 23,\n",
       "             'で': 24,\n",
       "             'です': 25,\n",
       "             'に': 26,\n",
       "             'に対して': 27,\n",
       "             'は': 28,\n",
       "             'まし': 29,\n",
       "             'クラス': 30,\n",
       "             'ネガティブ': 31,\n",
       "             'ポジティブ': 32,\n",
       "             'モデル': 33,\n",
       "             'レビュー': 34,\n",
       "             '値': 35,\n",
       "             '取り組み': 36,\n",
       "             '商品': 37,\n",
       "             '女性': 38,\n",
       "             '女王': 39,\n",
       "             '好き': 40,\n",
       "             '姫': 41,\n",
       "             '構築': 42,\n",
       "             '機械学習': 43,\n",
       "             '王': 44,\n",
       "             '王子': 45,\n",
       "             '男性': 46,\n",
       "             '短い': 47,\n",
       "             '自然言語処理': 48})"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "100%|█████████▉| 350697/351122 [01:08<00:00, 7133.80it/s]\u001b[A"
     ]
    }
   ],
   "source": [
    "# ベクトル化したバージョンのボキャブラリーを作成します\n",
    "TEXT.build_vocab(train_ds, vectors=japanese_fasttext_vectors, min_freq=1)\n",
    "\n",
    "# ボキャブラリーのベクトルを確認します\n",
    "print(TEXT.vocab.vectors.shape)  # 52個の単語が300次元のベクトルで表現されている\n",
    "TEXT.vocab.vectors\n",
    "\n",
    "# ボキャブラリーの単語の順番を確認します\n",
    "TEXT.vocab.stoi\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "女王 tensor(0.3650)\n",
      "王 tensor(0.3461)\n",
      "王子 tensor(0.5531)\n",
      "機械学習 tensor(0.0952)\n"
     ]
    }
   ],
   "source": [
    "# 姫 - 女性 + 男性 のベクトルがどれと似ているのか確認してみます\n",
    "import torch.nn.functional as F\n",
    "\n",
    "# 姫 - 女性 + 男性\n",
    "tensor_calc = TEXT.vocab.vectors[41] - \\\n",
    "    TEXT.vocab.vectors[38] + TEXT.vocab.vectors[46]\n",
    "\n",
    "# コサイン類似度を計算\n",
    "# dim=0 は0次元目で計算してくださいという指定\n",
    "print(\"女王\", F.cosine_similarity(tensor_calc, TEXT.vocab.vectors[39], dim=0))\n",
    "print(\"王\", F.cosine_similarity(tensor_calc, TEXT.vocab.vectors[44], dim=0))\n",
    "print(\"王子\", F.cosine_similarity(tensor_calc, TEXT.vocab.vectors[45], dim=0))\n",
    "print(\"機械学習\", F.cosine_similarity(tensor_calc, TEXT.vocab.vectors[43], dim=0))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "姫 - 女性 + 男性　を計算すると狙った通り、王子がもっとも近い結果になりました"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
